Different users may submit the same query to an information retrieval system and yet have different intents or meanings for their query. For example the query “jaguar” may be input with the intention of finding information about Jaguar cats or about luxury motor vehicles. Also, queries such as “machine learning” often conceal multiple intents: users may be looking for an overview of modern techniques, downloadable tools or theoretical results. In addition, queries which use acronyms are ambiguous because the acronym may stand for many different things. Real users also often mistype queries. However, information retrieval systems often have little or no mechanism for determining query intent and thus the information retrieved by the system is not relevant to the query input by the user.
Web search systems are an example of one type of information retrieval system. Users submit a query to the web search system and obtain a list of results that are relevant to the entered query. However, web search queries are often ambiguous or unclear. Different users may have different intents for the same query. The results that are returned may not be relevant to the query intent of a particular user.
In order to evaluate information retrieval systems human judges are often used to infer the intent of a query and then to judge how relevant a document is to that intent. However, it is difficult, even for human users, to infer the intent of queries which are ambiguous or unclear. This is because the human user needs to be familiar with a huge range of topics, cultures and motivations in order to infer query intent.
There is an ongoing need to improve the relevance of items retrieved by information retrieval systems such as web search systems. The embodiments described below are not limited to implementations which solve any or all of the disadvantages of known information retrieval systems.